feat(shadow): record validated value age on shadow match and mismatch - #136

What

When shadow validation delivers a match or mismatch verdict, DialCache now records the age of the validated cached value — the observing process's epoch clock minus the served frame's createdAtMs, in seconds, clamped at zero. A confirmed mismatch age answers the question this feature exists for: how long had the stale value been readable when validation caught it? The match series is the baseline that makes the mismatch histogram interpretable.

• New optional adapter hook observeShadowValueAge(labels, seconds) with the existing shadow labels (cacheNamespace, useCase, keyType, outcome). It does not gate shadow eligibility; only shadowValidation does.
• Prometheus: dialcache_shadow_value_age_histogram with buckets spanning 1s to 7d.
• Datadog: dialcache.shadow.value_age (distribution or histogram per the configured observation type).
• Outcomes that deliver no verdict on a retained value (superseded, filled, every error, timeout) record no age.

How (breaking)

Every Redis frame already carried an 8-byte createdAtMs header — Redis server time for tracked writes (stamp script), the writer's informational client clock for untracked ones — but both decoders stripped it before it could cross the client boundary.

BREAKING CHANGE: decodeRedisFrame, decodeTrackedRedisFrame, and DialCacheRedisClient.read() now return DecodedRedisFrame | null ({ payload, createdAtMs }) instead of the bare payload. The bundled node-redis and Valkey GLIDE adapters pass decoder results through and needed no code changes; custom clients must return the decoded frame. Per the pre-1.0 release policy this ships as a minor version.

Design notes:

• Watermark fencing is unchanged and still happens inside decodeTrackedRedisFrame; createdAtMs is consumed only for observability.
• The mismatch confirmation (C1) equality check remains byte-equality on payloads — identical bytes rewritten by a concurrent writer still confirm the mismatch verdict, exactly as before.
• Age uses the epoch clock (Date.now()), since frame stamps are epoch-based. It mixes clocks (server vs. client), so negative skew clamps to zero and the metric is documented as coarse operational evidence.
• The age is captured at verdict time inside the shadow flight; the retained-frame release path (flight.cachedFrame = null) is unchanged, and the packaged GC test still proves a timed-out flight releases its payload.

Validation

• pnpm typecheck ✓
• pnpm test — 516/516 (new coverage: match/mismatch age with pinned clocks, future-stamp clamp to zero, no age on superseded/filled, both metric adapters, decoder timestamp round-trips)
• pnpm build ✓
• pnpm test:package ✓ (packaged ESM/CJS round-trip assertions updated to the frame shape)
• pnpm test:integration — 139/139 against real Redis + Redis Cluster via testcontainers, including an end-to-end assertion that real shadow runs emit exactly two age observations (match, mismatch) and none for superseded

Migration notes for custom Redis clients

Nothing stored in Redis changes: the frame wire format, stamps, and fencing are byte-identical, and mixed 0.19/0.20 fleets interoperate in both directions. The break is confined to the in-process contract.

• Bundled adapters (createNodeRedisDialCacheClient, createValkeyGlideDialCacheClient): no action beyond the version bump.
• Custom DialCacheRedisClient implementations / direct decoder callers: read() and the dialcache/redis-protocol decoders now return DecodedRedisFrame | null ({ payload, createdAtMs }). TypeScript surfaces this as a compile error. Two plain-JS failure modes to know:
  • Returning the bare payload → every remote read fails open to a miss with serialization_load error metrics (loud, correctness-safe).
  • Returning { payload } without createdAtMs → serving and shadow validation work normally; only the value-age observation is affected, and DialCache skips recording non-finite ages rather than forwarding them to the metrics backend.
• Untracked writers must stamp real client time (Date.now()), not a constant — the stamp now feeds observeShadowValueAge on shadow verdicts.

Review

Ran a six-lane adversarial review (correctness, tests, simplicity, architecture, contracts, reliability + two-stage holistic audit) against cf087d0; four low-severity findings, all addressed in the follow-up commit: stale "untracked reads never consult it" contract docs, a Number.isFinite guard so an out-of-contract client stamp cannot poison backend histogram sums, a divergent-stamp confirmation test pinning mismatch-age provenance to the original frame, and a Prometheus histogram sum assertion.

🤖 Generated with Claude Code